Stick To The Shell

Stick To The Shell
Over the course of these tutorials we`ve been gradually adding more applications to make the NetBSD environment more user friendly and more like that of most other desktop operating systems. However no matter how hard we try it`s almost impossible to move away from using the shell completely. This is especially true from a system administrators point of view. For those of you that want to make a career in computing but dread the thought of using Windows all day Unix provides the perfect answer. Those of you who have less ambitious plans and simply want a Unix environment at home will still have to perform some administrative tasks from time to time, which invariably means using the shell. NetBSD comes supplied with two shells pre-installed, "sh", the basic shell and "csh" the c-shell which most of you will be using. While csh is fine for day to day use it is very difficult to write scripts in, a point we`ll cover later. On AFCD22 we've provided three new shells, tcsh is similar to the csh shell but features tab completion. zsh (the Z shell) and bash (the Borne Again SHell) have more in common with the korn shell and also both feature tab completion. Of course if you don't have a copy of the CD you can download the source code from the internet, for more information click here.

Each of the shells has it`s own strengths and weaknesses. c-shell variants are easy to use for day to day tasks whereas korn shell style shells tend to be more suited to slightly more complex commands. Korn shell programming is also usefull to know because some system files must be written in korn notation.
tcsh and zsh are provided pre-complied on the CD with installation scripts. Only the source code for bash is on this months CD so you will have to compile it first. Just to make things tricky I`m going to be using bash for the rest of this article, although you can probably use zsh for most of the examples.

How To Speak NetBSD Guide To Pronounciation
cshrc ~ #! etc usr	kush rook twiddle hash bang et sea user

A fun guide to how to say certain unix phrases to make you sound like a Unix God.

As you are using the shells you may find certain aspects of them that you wish to change slightly. Each shell has a configuration file which you can use to tailor your working environment to your needs. For bash and zsh users life is simple. There is only one config file to worry about. For csh and tcsh users you have 3 files to configure. The first file is the .login file. This is read every time you log into a machine. It is recommended that you define any Environment Variables here, as they are automatcially passed to sub shells, and also include any commands that you wish to run every time you log in, for example to check for mail and news. The next file is the .cshrc, this file is read every time you start a shell and should contain shell variables like prompt and any aliases you might require. Finally there is a .logout file which is executed when you close a login shell, if you want to run any commands when you logout this is the place to put them.
If this all sounds a bit confusing don`t worry as there are example configuration files here.

As I've said before Unix is a very powerful operating system, and most of this power can only be accessed thorugh the shell. The most common way of doing this is to write shell scripts. These are text files that contain a list of instructions that the shell executes. The only difference between writing a script and typing the commands into the shell itself is that you only need to type a script once. The power of the shells lies in the way commands can interact with each other. There are 2 main ways this occures. The first, and most commonly used is a pipe (|). When two, or more commands are seperated by a pipe symbol the output of the first command is fed into the second command. For example the command "ls -l | wc -l" will tell you how many files are in a directory. Why, well ls -l produces a directory listing with one file per line, and wc -l counts the number of lines in a file. By combining them using a pipe, wc can be used to count the number of lines in a directory listing, and thus the number of files.
The second way commands interact is for one command to execute another command as part of it`s arguments. e.g. "cd `echo /`", which is a long winded way of changing to the root directory!

A Simple Script

In order to see exactly what we can achieve with the shell we'll look at two example scripts. Both are simple shell scripts that perform tasks that it would be difficult to do otherwise. The first script (listing 1) converts files from uppercase to lowercase, and uses pipes, nested commands and loops. The first line in any shell script tells NetBSD which shell to use to interpret the script. This does not have to be the same as the shell you are currently using. Lines 2,3 and 5 for a for loop. The shell will scan the current directory for any files ending in .txt. For each file that it finds it will assign the value of the varibale $f to that file name and execute the contents of the loop, between the "do" and the "done". Line 4 forms the body of the loop and does the real work. If you look closely you will see that it is nothing more complicated that a simple mv command.

Listing 1 - up2lower.sh

#!/bin/sh



for f in *.txt 

do

mv $f `echo $f | tr [A-Z] [a-z]`

done

The first argument of the mv command is $f, which for each pass of the loop contains the name of a .txt file. The second argument is slightly more complicated. The commands enclosed by back quotes are executed each time around the for loop, and the result forms th e second argument to the move command. To understand how these commands interact with each other it is helpfull to see each part of the script in action. The first argument to the mv command can be found by typing the following....




for f in *.txt

do

echo $f

done

which will display all .txt files in the directory. To see what the second argument to mv will be use the following command



echo HELLO | tr [A-Z] [a-z]

which will convert HELLO into hello.
By combining both these scripts, we are able to convert file names from upper to lower case.

The second script, which produces an error message if any filesystem is more than 97% full is slightly more complicated, but it should be easy to figure out what is really going on if you examine each command seperatly (use the man command if you need more information about a certain command). One of the more complex NetBSD commands sed (Stream EDitor) is used in this script. It simply removes the % sign from the value for the disk capacity, so it is easier to compare against another number. Whilst the first script is designed to be run only occasionally when the need arise to convert files, the second script should be run on a more regular basis. It may be that you want to run it as part of your login script, however if your machine is always running you may not log in and out very often.

Listing 2 - fscheck.sh

#!/bin/sh

PATH=/bin:/usr/bin

df -k -F ufs | tail +2 | while read disk zz zz zz cap mount
do
if test `echo $cap | sed -e "s/%/ /g"` -gt 97
then
echo "$mount filesystem almost full"  
fi
done

To make sure that the script is run on a regular basis you should make use of NetBSD`s scheduler "cron", which provides a mechanism for executing commands at set times or intervals. The cron information is stored in a file known as a crontab which can be manipulated with the crontab command. Be carefull though as simply typing crontab will destroy any existing entries. You should use crontab -l to list the content of the file and crontab -e to edit it. The crontab -e command uses the EDITOR environment variable to determine which editor to invoke, and will default to vi if this variable is not set. While it is possible to create crontab entries for each individual user, administrative tasks, such as our diskspace script would normally go in the root crontab. Each time cron executes a command it usually emails the output of that command to the user that ran it. For commands running as root it is usually more convientient to email the output to a normal user. In the case of our disk space checking script this would only require minor changes. The most elegant solution would be to change line 6 to read



echo "$mount filesystem almost full" | mail chrisl

in order to stop crontab also mailing the output of this script to root add ">/dev/null 2>&1" to the end of the command. This redirects the standard output and standard error produced by the script to /dev/null, NetBSD`s trash can.
Another task many systems administrators perform regularly is monitoring what processes are running on the machine. Usually this is to investigate a loss in perfomance. As a typical NetBSD system can have many users and also runs many programs in the background such as networking daemons it can be difficult to determine exactly what the processor is doing. NetBSD comes complete with commands such as ps, which will list the processes running on a machine and vmstat which can be used to give a running report on disk, memory and cpu usage, however to see which processes are taking up valuable cpu time an external command such as "top" must be used.

As you can see top is right at the top itself!
Top is included (pre-compiled) on this months CD. When run it will show you the top processes, in terms of cpu utilisation. Top is notorious amongst Unix users for always appearing near the top of this list itself! Many of the other common administrative such as adding users and installing applications you will already have performed as part of the NetBSD installation and previous tutorials, and you will find NetBSD has a large array of commands that you can use to create your own scripts. If you want to learn more about scripting, shells and Unix in general you will find a book such as "Unix Power Tools" by Jerry Peek, Tim O Reilly and Mike Loukides pub O'Reilly/Random House indispensible.

In the next article we'll be looking at installing and configuring Apache and Perl.